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Abstract. In this paper, we describe a brand new key exchange pro- 
tocol based on a semidirect product of (semi)groups (more specifically, 
^ | on extension of a (semi)group by automorphisms), and then focus on 

practical instances of this general idea. Our protocol can be based on 
any group, in particular on any non-commutative group. One of its spe- 
cial cases is the standard Dime-Hellman protocol, which is based on a 
cyclic group. However, when our protocol is used with a non-commutative 
(semi)group, it acquires several useful features that make it compare fa- 
vorably to the Dime-Hellman protocol. Here we also suggest a particular 
non-commutative semigroup (of matrices) as the platform and show that 
security of the relevant protocol is based on a quite different assumption 
compared to that of the standard Dime-Hellman protocol. 
O 
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1 Introduction 

j^ | It is rare that the beginning of a whole new area of science can be traced back to 

J-j ■ one particular paper. This is the case with public key cryptography; it started 

with the seminal paper [2]. 

The simplest, and original, implementation of the protocol uses the multi- 
plicative group of integers modulo p, where p is prime and g is primitive mod p. 
A more general description of the protocol uses an arbitrary finite cyclic group. 
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1. Alice and Bob agree on a finite cyclic group G and a generating element g 
in G. We will write the group G multiplicatively. 

2. Alice picks a random natural number a and sends g a to Bob. 

3. Bob picks a random natural number b and sends g b to Alice. 

4. Alice computes K A = (g h ) a = g ba ■ 

5. Bob computes K B = (g a ) b = g ab ■ 



Since ab = ba, both Alice and Bob are now in possession of the same group 
element K = Ka = Kb which can serve as the shared secret key. 

The protocol is considered secure against eavesdroppers if G and g are chosen 
properly. The eavesdropper must solve the Diffie-Hellman problem (recover g ab 
from g, g a and g b ) to obtain the shared secret key This is currently considered 
difficult for a "good" choice of parameters (see e.g. [5] for details). 

There is an ongoing search for other platforms where the Diffic-Hcllman or 
similar key exchange could be carried out more efficiently, in particular with 
public/private keys of smaller size. This search already gave rise to several inter- 
esting directions, including a whole area of elliptic curve cryptography. We also 
refer the reader to [3] for a survey of proposed cryptographic primitives based 
on non-abelian (= non-commutative) groups. A survey of these efforts is outside 
of the scope of the present paper; our goal here is to suggest a new key exchange 
protocol based on extension of a (semi)group by automorphisms. Our protocol 
can be based on any group, in particular on any non-commutative group. It has 
some superficial resemblance to the classical Diffie-Hellman protocol, but there 
are several distinctive features that, we believe, give our protocol important ad- 
vantages. In particular, even though the parties do compute a large power of a 
public element (as in the classical Diffic-Hcllman protocol) , they do not transmit 
the whole result, but rather just part of it. 

We also describe in this paper some particular instances of our general pro- 
tocol. In particular, we suggest a non-commutative semigroup (of matrices) as 
the platform and show that security of the relevant protocol is based on a quite 
different assumption compared to that of the standard Diffic-Hcllman protocol. 

We mention another, rather different, proposal [8] of a cryptosystem based on 
the semidirect product of two groups and yet another, more complex, proposal 
of a key agreement based on the semidirect product of two monoids [I]. Both 
these proposals are very different from ours. Also, the extended abstract [3], 
despite the similarity of the title, has very little overlap with the present paper. 
In particular, the key exchange protocol in Section [3] of the present paper is 
brand new. 

Finally, we note that the basic construction (semidirect product) we use in 
this paper can be adopted, with some simple modifications, in other algebraic 
systems, e.g. associative rings or Lie rings, and key exchange protocols similar 
to ours can be built on those. 



2 Semidirect products and extensions by automorphisms 

We include this section to make the exposition more comprehensive. The reader 
who is uncomfortable with group-theoretic constructions can skip to subsection 

We now recall the definition of a semidirect product: 

Definition 1. Let G,H be two groups, let Aut(G) be the group of automor- 
phisms of G, and let p : H — s> Aut{G) be a homomorphism. Then the semidirect 
product of G and H is the set 

r = G-A p H = {(g, h):geG, h e H] 

with the group operation given by 

{g,h){g',h') = (g"^-g', h-h'). 
Here g p( - h ' denotes the image of g under the automorphism p(h'), and when we 
write a product h-h' of two morphisms, this means that h is applied first. 

In this paper, we focus on a special case of this construction, where the 
group H is just a subgroup of the group Aut(G). If H = Aut(G), then the 
corresponding semidirect product is called the holomorph of the group G. We 
give some more details about the holomorph in our Section 12.11 and in Section 
Owe describe a key exchange protocol that uses (as the platform) an extension 
of a group G by a cyclic group of automorphisms. 

2.1 Extensions by automorphisms 

A particularly simple special case of the semidirect product construction is where 
the group H is just a subgroup of the group Aut(G). If H = Aut(G), then the 
corresponding semidirect product is called the holomorph of the group G. Thus, 
the holomorph of G, usually denoted by Hol(G), is the set of all pairs (g, (f>), 
where g <G G, <f> S Aut(G), with the group operation given by (g, <j>) ■ (g' , cf>') = 
Wig) -g', </>■</>'). 

It is often more practical to use a subgroup of Aut{G) in this construction, 
and this is exactly what we do in Section [3l where we describe a key exchange 
protocol that uses (as the platform) an extension of a group G by a cyclic group 
of automorphisms. 

Remark 1. One can also use this construction if G is not necessarily a group, 
but just a semigroup, and/or consider endomorphisms of G, not necessarily au- 
tomorphisms. Then the result will be a semigroup; this is what we use in our 
Section [5] 



3 Key exchange protocol 

In the simplest implementation of the construction described in our Section 12 .11 
one can use just a cyclic subgroup (or a cyclic subsemigroup) of the group Aut(G) 



(respectively, of the semigroup End{G) of endomorphisms) instead of the whole 
group of automorphisms of G 

Thus, let G be a (semi)group. An element g £ G is chosen and made public as 
well as an arbitrary automorphism <j) 6 Aut(G) (or an arbitrary endomorphism 
4> £ End(G)). Bob chooses a private n € N, while Alice chooses a private m e N. 
Both Alice and Bob are going to work with elements of the form (g, 4> r ), where 
g £ G, r e N. Note that two elements of this form are multiplied as follows: 
(g^ r )-(h,r) = ((t> s (g)-h 1 ^ +s ). 

1. Alice computes (g,4>) m = (0 m_1 (<?) • • ■ </> 2 (,9) ■ </>(g) • <?, <fi m ) and sends only 
the first component of this pair to Bob. Thus, she sends to Bob only the 
element a = f/)™ -1 ^) • • • (j) 2 {g) ■ <j)(g) ■ g of the (semi)group G. 

2. Bob computes (g,<f>) n — (<j> n ~ 1 (g) • • ' < / ,2 (ff) '^{g) 'Si 0") and sends only the 
first component of this pair to Alice. Thus, he sends to Alice only the 
element b = (j) n ~ 1 (g) ■ ■ ■ (j) 2 (g) ■ <f>(g) ■ g of the (semi)group G. 

3. Alice computes (b,x) ■ (a, cf> m ) = (cf> m (b) ■ a, x ■ (j> m ). Her key is now Ka = 
</> m (6) • a. Note that she does not actually "compute" x ■ (\> m because she does 
not know the automorphism x = <p n ; recall that it was not transmitted to 
her. But she does not need it to compute Ka- 

4. Bob computes (a, y)- (6, </>") = (<j> n (a)-b, y-(f> n ). His key is now K B = cj) n (a)-b. 
Again, Bob does not actually "compute" y ■ cf> n because he does not know 
the automorphism y = <p m . 

5. Since (b,x) ■ (a, (j) m ) = (a, y) ■ (6, <p n ) = (g, 4>) m+n , we should have K A = 
Kb = K, the shared secret key. 

Remark 2. Note that, in contrast with the "standard" Diffic-Hcllman key ex- 
change, correctness here is based on the equality h m ■ h n — h n ■ h m = h m+n 
rather than on the equality {h m ) n = (h n ) m = h mn . In the "standard" Diffie- 
Hcllman set up, our trick would not work because, if the shared key K was just 
the product of two openly transmitted elements, then anybody, including the 
eavesdropper, could compute K. 

4 Computational cost 

From the look of transmitted elements in our protocol in Section [31 it may seem 
that the parties have to compute a product of m (respectively, n) elements of 
the (semi)group G. However, since the parties actually compute powers of an 
element of G, they can use the "square-and-multiply" method, as in the standard 
Diffic-Hcllman protocol. Then there is a cost of applying an automorphism </> to 
an element of G, and also of computing powers of <f>. These costs depend, of 
course, on a specific platform (semi)group that is used with our protocol. In our 
first, "toy" example (Section [5] below), both applying an automorphism <j> and 
computing its powers amount to exponentiation of elements of G, which can 



be done again by the "square-and-multiply" method. In our main example, in 
Section [51 (j> is a conjugation, so applying amounts to just two multiplications 
of elements in G, while computing powers of 4> amounts to exponentiation of two 
elements of G (namely, of the conjugating element and of its inverse). 

Thus, in either instantiation of our protocol considered in this paper, the cost 
of computing (g, <fi) n is 0(log n), just as in the standard Diffic-Hellman protocol. 



5 "Toy example": multiplicative Z* 

As one of the simplest instantiations of our protocol, we use here the multiplica- 
tive group Z* as the platform group G to illustrate what is going on. In selecting 
a prime p, as well as private exponents m, n, one can follow the same guidelines 
as in the "standard" Diffic-Hellman. 

Selecting the (public) endomorphism <p of the group Z* amounts to selecting 
yet another integer k, so that for every h £ Z* one has <j)(h) = h k . If k is 
relatively prime to p — 1, then </> is actually an automorphism. Below we assume 
that k > 1. 

Then, for an element g £ Z* we have: 

( 5 ,0r = («r- 1 (s)---<Msw 2 (s)-.9, <n- 

We focus on the first component of the element on the right; easy computation 
shows that it is equal to g k +-+fc+i _ g k _ x _ Thus, if the adversary chooses 
a "direct" attack, by trying to recover the private exponent m, he will have to 
solve the discrete log problem twice: first to recover k _ 1 from g k - 1 , and then 
to recover m from k m . (Note that k is public since <f> is public.) 

On the other hand, the analog of what is called "the Diffic-Hellman problem" 

would be to recover the shared key K = g k - 1 from the triple (g, g k - 1 , g k - x 



Since g and k are public, this is equivalent to recovering g k from the triple 
(Si 9 k i 9 k )> i- e -> this is exactly the standard Dimc-Hcllman problem. 

Thus, the bottom line of this example is that the instantiation of our protocol 
where the group G is Z* is not really different from the standard Diffic-Hellman 
protocol. In the next section, we describe a more interesting instantiation, where 
the (semi)group G is non-commutative. 



6 Matrices over group rings and extensions by inner 
automorphisms 

To begin with, wc note that our general protocol in Section [3] can be used 
with any non-commutative group G if <f> is selected to be a non-trivial inner 
automorphism, i.e., conjugation by an element which is not in the center of G. 
Furthermore, it can be used with any non-commutative semigroup G as well, 
as long as G has some invertible elements; these can be used to produce inner 



automorphisms. A typical example of such a semigroup would be a semigroup 
of matrices over some ring. 

In the paper [3], the authors have employed matrices over group rings of a 
(small) symmetric group as platforms for the (standard) Difhc-Hellman-like key 
exchange. In this section, we use these matrix semigroups again and consider 
an extension of such a semigroup by an inner automorphism to get a platform 
semigroup for our protocol. 

Recall that a (semi)group ring R[S] of a (semi)group S over a commutative 
ring R is the set of all formal sums 

g.es 

where r^ £ R, and all but a finite number of r, are zero. 
The sum of two elements in R[G] is defined by 




The multiplication of two elements in R[G] is defined by using distributivity. 

As we have already pointed out, if a (scmi)group G is non-commutative 
and has non-central invertible elements, then it always has a non-identical inner 
automorphism, i.e., conjugation by an element g € G such that g~ 1 hg ^ h for 
at least some h £ G. 

Now let G be the semigroup of 3 x 3 matrices over the group ring ^[A^, 
where A§ is the alternating group on 5 elements. Here we use an extension of the 
semigroup G by an inner automorphism ip H , which is conjugation by a matrix 
H £ GL^CE^A^]). Thus, for any matrix M £ G and for any integer k > 1, we 
have 

<p H (M) = H^MH; (p k H {M) = H- k MH k . 
Now our general protocol from Section [3] is specialized in this case as follows. 

1. Alice and Bob agree on public matrices M £ G and H £ GL^CZjIA^}). Alice 
selects a private positive integer m, and Bob selects a private positive integer 
n. 

2. Alice computes (M, ip H ) m = (H- m+1 MH m - 1 ■ ■ ■ H~ 2 MH 2 -H- 1 MH-M, <p™) 
and sends only the first component of this pair to Bob. Thus, she sends 
to Bob only the matrix 

A = H- m+1 MH m - 1 ■ ■ ■ H~ 2 MH 2 ■ H^MH ■ M = IJ- m (HM) m . 



3. Bob computes (M, <p H ) n = (H- n+1 MH n - 1 ■ ■ ■ H^MEP-H^MH-M, Lp n H ) 
and sends only the first component of this pair to Alice. Thus, he sends 
to Alice only the matrix 

B = H- n+1 MH n - 1 ■ ■ ■ H- 2 MH 2 ■ H~ l MH ■ M = H~ n {HM) n . 

4. Alice computes (B,x) ■ (A, tp™) = (cp™(B) ■ A, x ■ (f™)- Her key is now 
K AHce = tp™(B) ■ A = H-( m+n \HM) m+n . Note that she docs not actually 
"compute" x ■ (p™ because she does not know the automorphism x = <p n H ; 
recall that it was not transmitted to her. But she does not need it to compute 

-"• Alice- 

5. Bob computes (A, y)-(B, <p n H ) = (<p% (A) B, y<p n H )- His key is now K Bo b = 
<p™ (A) ■ B. Again, Bob does not actually "compute" y ■ ip n H because he does 
not know the automorphism y = (p™. 

6. Since (B,x) ■ (A, tp™) = (A, y) ■ (B, tp n H ) = (M, (p H ) m+n , we should have 
K Alice = KBob = K, the shared secret key. 



7 Security assumptions and analysis 

In this section, we address the question of security of the particular instantiation 
of our protocol described in Section [5] 

Recall that the shared secret key in the protocol of Section [5] is 

K = (p™(B) ■ A = Lp n H {A) ■ B = H- ( ~ m+n \HM) m+n . 

Therefore, our security assumption here is that it is computationally hard to 
retrieve the key K = iJ-( m +»)(#A/)™+™ f rom the quadruple 
(H, M, H- m (HM) m , H- n {HM) n ). 

In particular, we have to take care that the matrices H and HM do not com- 
mute because otherwise, K is just a product of H~ m (HM) m and H~ n (HM) n . 

A weaker security assumption arises if an eavesdropper tries to recover a pri- 
vate exponent from a transmission, i.e., to recover, say, m from H~ m {HM) m . 
A special case of this problem, where H = I, is the "discrete log" problem 
for matrices over Z^[A^}, namely: recover m from M and M m . Even this prob- 
lem appears to be hard; it was addressed in [J] in more detail. In particular, 
statistical experiments show that for a random matrix M, matrices M m are 
indistinguishable from random. 

In order to verify the robustness and security of our protocol, we have ex- 
perimentally addressed two questions. The first question is whether or not any 
information about the private exponent n is leaked from transmission. That is, 
for a random exponent n, how different is the matrix (Al,ip H ) n from N, where 
N is random? The second point that needs verification is to determine how dif- 
ferent the final shared key is from a random matrix. More specifically, if Alice 
and Bob choose secret integers m and n respectively, how different is the matrix 
(M, ip H ) n+m from (M, f H ) q , where q is of the same bit size are n + m. 



To perform the first experimental validation we worked over M^{'L'j[A^\) and 
used random choices of n G [fO 44 , 10 55 ]. We then looked at the two distributions 
generated by the first component of (M,ip H ) n and N, where M and N are 
random matrices. We need to verify that the two generated distributions are in 
fact indistinguishable. To this end we looked at the components of each matrix 
and counted the frequency of occurrence of each element of A 5 . We repeated 
this process 500 times and generated a frequency distribution table for the two 
distributions. 

From the table, we produced Q — Q (quantile) plots of the entries of the two 
matrices: the first component of (M,(p H ) n and a random matrix N. Quantile 
plots are a quick graphical tool for comparing two distributions. These plots 
essentially compare the cumulative distribution functions of two distributions. 
If the distributions are identical, the resulting graph will be a straight line. 
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Fig. 1. Results for M" vs. N 



Figure [T] shows the resulting plots for this experiment. These graphs show 
that the two distributions are in fact identical, therefore suggesting that no 
information about a private exponent n is revealed by transmissions between 
Alice and Bob. 

The second experiment wc carried out was similar to the first one, except in 
this case we were comparing the first components of (M, (fi H ) n and (M, (p H ) a+b , 
where n, a and b are random and all of roughly the same bit size, i.e. all are 
integers from [10 , 10 55 ]. This experiment helps address the DDH (decisional 
Diffie-Hellman) assumption by comparing the shared secret key to a random key 
and ensuring that no information about the former is leaked. See Figure [5] for 
the resulting Q — Q plots. These 9 graphs suggest that the two distributions 
generated by these keys are in fact indistinguishable. 
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8 Parameters and key generation 

Private exponents m and n should be of the magnitude of 2*, where t is the 
security parameter, to make brute force search infeasible. Thus, m and n are 
roughly t bits long. 

Public matrix M is selected as a random 3x3 matrix over the group ring 
Z^lAs], which means that each entry of M is a random element of Zj[A^\. The 
latter means that each entry is a sum YIo-pa Ci 9i °^ elements of the group 
A§ with coefficients Cj selected uniformly randomly from Z7. Thus, although 
the bit complexity of the matrix M is fairly high (9 • 3 • 60 = 1620 bits), the 
procedure for sampling M is quite efficient. We want to impose one restriction on 
the matrix M though. There is a trivialization (sometimes called augmentation) 
homomorphism of the group ring that sends every group element to 1. This 
homomorphism naturally extends to a homomorphism of the whole semigroup of 
matrices. To avoid leaking any information upon applying this homomorphism, 
we want the image of every entry of M to be 0. Group ring elements like that 
are easy to sample: after sampling a random element YIo-pa Ci 9i °^ ^ 7 [-^5]' we 
select a random coefficient a and change it, if necessary, to have J^. Cj = 0. 

Note that with this choice of M, applying the trivialization homomorphism 
to any of the transmitted matrices in our protocol will produce the zero matrix, 
thus not leaking any information. We also note that there are no other homo- 
morphisms of the group A$ (which is a finite simple group), except for inner 
automorphisms. This will prevent an eavesdropper from learning partial infor- 
mation about secret keys by applying homomorphisms to transmitted matrices. 

Finally, we need to sample an invertible 3x3 matrix H over the group ring 
Z7 [A5] . There are several techniques for doing this; here we give a brief exposition 
of one possible procedure. 

We start with an already "somewhat random" matrix, for which it is easy to 
compute the inverse. An example of such a matrix is a lower/upper triangular 
matrix, with invertible elements on the diagonal: 

fgi ui u 2 N 
U = I g 2 u 3 
,0 <? 3 , 

Here gi are random elements of the group A$, and Ui are random elements 
of the group ring Z7L45]. We then take a random product, with 20 factors, of 
such random invertible upper and lower triangular matrices, to get our invertible 
matrix H. 

We note that there is always a concern (also in the standard Diffic-Hcllman 
protocol) about the order of a public element: if the order is too small, then a 
brute force attack may be feasible. In our situation, this concern is significantly 
alleviated by the fact that our transmissions are products of powers of two 
different matrices rather than powers of a single matrix. Therefore, even if the 
order of one of the matrices happens to be small by accident, this does not mean 
that the product H~ m (HM) m will go into loop of a small size. Furthermore, 



since our matrix M is non-invertible, it does not have an "order" , but rather a 
loop: M r = M a for some positive r =£ s. The matrices HM and H~ m (HM) m 
are non-invertible, too, so they do not have an order either, but rather a loop. 
Detecting a loop is, in general, computationally much harder than computing 
the order of an invertible element. 



9 Conclusions 

We have presented a brand new key exchange protocol based on extension of a 
(scmi)group by automorphisms and described some practical instances of this 
general idea. Our protocol can be based on any group, in particular on any non- 
commutative group. It has some superficial resemblance to the classical Diffic- 
Hcllman protocol, but there are several distinctive features that, we believe, give 
our protocol important advantages: 

• Even though the parties do compute a large power of a public element (as 
in the classical Diffic-Hellman protocol), they do not transmit the whole result, 
but rather just part of it. 

• Since the classical Diffic-Hellman protocol is a special case of our protocol, 
breaking our protocol even for any cyclic group would imply breaking the Diffic- 
Hellman protocol. 

• If the platform (semi)group is not commutative, then we get a new security 
assumption. In the simplest case, where the automorphism used for extension 
is inner, attacking a private exponent amounts to recovering an integer n from 
a product g~ n h n , where g,h are public elements of the platform (semi)group. 
In the special case where g = 1 this boils down to recovering n from h n , with 
public h ("discrete log" problem). 

On the other hand, in the particular instantiation of our protocol, which is 
based on a non-commutative semigroup extended by an inner automorphism, 
recovering the shared secret key from public information is based on a different 
security assumption than the classical Diffie-Hellman protocol is. Namely, the 
assumption is that it is computationally hard to retrieve the shared secret key 
K = fi-( m + n ) g m + n from the triple of elements (h, h~ m g m , h~ n g"), assuming 
that g and h do not commute. 
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